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Figure  1 .  The  intimidating  complexity  of  the  F-4  cockpit  designed  in  the  late  1950 \ s  typifies 
the  single-function  mechanical  controls  and  displays .  Displays  had  to  be  small  to  get  so 
many  on  the  panel.  All  the  information  available  is  displayed  to  the  pilot  all  the  time.  Only 
a  few  of  these  intercommunicate.  Photo  taken  at  the  U.S.  Air  Force  Museum,  Dayton,  OH, 
by  Larry  Burgess,  University  of  Dayton.  Digital  photo-imaging  by  David  W.  Radabaugh. 


Obsolete  Accounting  Model 
Hinders  Crew  System  Integration 


Joe  W.  McDaniel 


DISTRIBUTION  STATEMENT  A 

Approved  for  Public  Release 


Editor’s  note:  The  views  presented  are 
those  of  the  author  and  do  not  neces¬ 
sarily  represent  the  views  of  the  DOD  or 
its  components.  ]AL 

Ouring  the  last  30  years,  the 
complexity  of  aircraft- 
cockpits  has  evolved  be¬ 
yond  the  range  of  tradi¬ 
tional  management  technology.  To¬ 
day,  multifunction  digital  controls/ 
displays,  multiple  interconnected 
processors,  and  the  need  for  a  truly 
integrated  crew  system  create  engi¬ 


nes'  mat;  are  not  being 
effectively  met.  To  be  effective,  the 
modern  crew  system  must  be  inte¬ 
grated,  consistent,  and  compatible  with 
the  capabilities  of  the  operator. 
Notably,  the  design  and  program  man¬ 
agement  environment  must  also 
evolve  commensurate  with  the  system 
being  developed.  In  other  words,  an 
integrated  design  process  is  necessary 
to  develop  an  effectively  integrated 
system. 

Failure  of  some  of  the  design  sup- 
Continued  on  page  2 


port  processes  to  evolve  to  meet  the 
demands  of  the  digital  crew  system 
have,  in  some  cases,  actually  become 
a  hindrance  to  effective  design.  One 
serious  impediment  to  integrating  crew 
system  functions  may  be  the  aircraft 
model  in  Appendix  A  of  MIL-STD- 
881B  Work  Breakdown  Structure  For 
Defense  Materiel  Items  (WBS).  The 
WBS  is  prescribed  for  use  on  new 
system  acquisitions  to  aid  definition, 
analysis,  tracking,  and  control  of  each 
element  of  the  system  throughout 
development.  The  WBS  is  a  hierarchi¬ 
cal  diagram  that  decomposes  the 
entire  system  into  elements, 
subelements,  sub-sub,  etc.,  down  to 
the  level  of  each  element  of  hardware, 
software,  services,  data,  training,  sup¬ 
port  equipment,  management,  and 
other  work  tasks. 

This  is  not  meant  to  attack  either 
military  standards  or  the  WBS  concept, 
but  rather  a  specific  part  of  one  that 
has  become  obsolete.  The  WBS  is 
absolutely  necessary  for  developing  a 
complex  system.  If  we  did  not  already 
have  the  WBS  process,  one  would 
have  to  be  invented.  The  WBS  pro¬ 
vides  a  consistent  mechanism  for  track¬ 
ing  all  the  subcontracts  and  suppliers 
contributing  to  the  system.  Its  most 
important  function  is  in  tracking  the 
cost,  schedule,  and  progress  of  each 
element.  The  problem  is  simply  that 
the  model  WBS  for  aircraft  does  not 
include  an  element  for  the  crew  sys¬ 
tem.  That  model  was  developed  in  the 
early  1970s.  When  this  standard  was 
last  updated  (March  1993),  this  defi¬ 
ciency  remained  uncorrected. 

A  brief  review  will  illustrate  the 
problem.  In  the  WBS  hierarchical 
model  for  an  aircraft,  Level  1  has  but  a 
single  element,  the  entire  Aircraft  Sys¬ 
tem.  The  ten  Level-2  elements  listed  in 
Table  1  point  out  that  an  aircraft  sys¬ 
tem  is  much  more  than  an  aircraft.  The 
system  includes  training  and  trainers; 
it  includes  hangars  and  mechanics;  it 
includes  everything  necessary  to  own 
and  operate  the  aircraft.  The  aircraft  is 
just  one  element  at  this  second  level  in 
the  WBS  hierarchy. 

The  problem  for  the  crew  system 


Table  1. 

The  model  hierarchy  for  Aircraft  Systems  in  Appendix  A  of 
MIL-STD-881B  has  10  Level-2  elements  under  the  Level-1  Aircraft 
System.  Notice  that  the  aircraft  (Air  Vehicle)  is  just  one  of  the  10 

elements. 


Air  Vehicle 

Systems  Engineering/Program  Management 

System  Test  and  Evaluation 

Training 

Data 

Peculiar  Support  Equipment 
Common  Support  Equipment 
Operational/Site  Activation 
Industrial  Facilities 
Initial  Spares  and  Repair  Parts 


Table  2. 

Under  Air  Vehicle,  there  are  17  Level-3  elements  in  the  model. 
The  crew  system  of  an  Air  Force  aircraft  is  scattered  among  the  12 
underlined  Level-3  elements. 


Airframe 

Propulsion 

Air  Vehicle  Applications  Software 

Air  Vehicle  System  Software 

Communications/Identification 

Navigation/Guidance 

Central  Computer 

Fire  Control 

Data  Display  and  Controls 
Survivability 
Reconnaissance 
Automatic  Flight  Control 
Central  Integrated  Checkout 
Antisubmarine  Warfare 
Armament 
Weapons  Delivery 
Auxiliary  Equipment 
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occurs  at  the  third  level  of  the  model 
hierarchy.  At  the  third  level,  the  Air 
Vehicle  itself  is  subdivided  into  the  17 
elements  listed  in  Table  2.  At  a  glance 
one  can  see  that  none  of  the  17  is  a 
cockpit,  a  crew  system,  or  any  equiva¬ 
lent  terminology.  The  Air  Force  air¬ 
craft  crew  system  is  scattered  among  at 
least  twelve  (underlined)  of  the  seven¬ 
teen  Level-3  elements. 

The  definitions  of  these  elements 
are  too  lengthy  to  be  reproduced  here, 
but  the  following  will  summarize  the 
dispersed  nature  of  the  crew  system. 
The  Airframe  includes  manual  flight 
controls,  fuel,  navigation,  and  engine 
displays,  but  the  Propulsion  includes 
the  engine  controls  (if  furnished  as  an 
integral  part  of  the  engine).  The  Air 
Vehicle  Applications  Software  relates 
to  pilot  controls  and  displays.  Com¬ 
munications/Identification  has  the  ra¬ 
dios  for  talking,  but  Navigation/Guid¬ 
ance  has  the  radios  for  range  and 
bearing,  radar,  compasses,  etc.,  ex¬ 
cept  for  the  terrain-following  radar, 
which  is  under  Survivability.  Central 
Computer  coordinates  and  directs 
some,  but  not  all  of  the  avionics  sys¬ 
tems,  and  its  software  is  in  a  separate 
element.  If  the  aircraft  is  a  warplane, 
the  Fire  Control  functions  have  a  sepa¬ 
rate  element.  The  Data  Display  and 
Controls  sounds  a  lot  like  a  cockpit, 
but  is  actually  a  miscellaneous  catch¬ 
all  that  includes  those  multifunction 
controls/displays  that  are  not  specifi¬ 
cally  defined  elsewhere. 

The  advocates  for  this  modular  struc¬ 
ture  point  to  the  different  missions  of 
various  aircraft.  For  example,  only 
combat  aircraft  need  Fire  Control  func¬ 
tions;  trainers  and  transports  do  not. 
System  planners  are  supposed  to  de¬ 
lete  any  elements  that  are  not  relevant. 
This  modular  approach  was  very  ap¬ 
propriate  when  the  WBS  process  was 
standardized  back  in  the  early  1970s. 
Then,  the  pilot’s  crew  station  was 
composed  of  several  independent  sub¬ 
systems,  usually  supplied  by  different 
subcontractors.  Then,  it  was  the  prime 
contractor’s  job  to  locate  each  of  these 
subsystems  in  the  aircraft.  In  the 
context  of  the  cockpit  design,  the 


prime  contractor’s  effort  centered  on 
the  cockpit  layout  and  installation  of 
controls  and  displays,  which  generally 
did  not  intercommunicate.  Figure  1 
(page  1)  shows  an  example  of 
mechanical  analog  instrumentation 
having  single-purpose  controls  and 
displays. 

In  contrast,  Figure  2  depicts  a  mod¬ 
ern  digital  cockpit  having  an  almost 
generic  physical  appearance,  clean  and 
uncluttered,  consisting  of  a  few  multi¬ 
function  controls  and  a  few  multifunc¬ 
tion  digital  displays.  Today,  the  critical 
design  issues  in  the  crew  system  relate 
to  information  management  and  inte¬ 
gration  of  data.  The  modern  crew 
system  is  no  longer  a  mere  collection 
of  subsystems,  but  a  highly  integrated 
information  and  control  system  in 
which  each  data  input  (sensor,  con¬ 
trol,  communication,  etc.)  is  not  just 
displayed  to  the  pilot,  but  is  fed  into  a 
common  digital  data  bus  to  allow 
additional  information  to  be  computed 
and  shared  throughout  the  system.  So 
much  information  is  now  available 
that  only  a  small  portion  can  be  shown 


to  the  pilot  at  any  one  time.  Integra¬ 
tion  of  the  entire  system  is  key  to 
modern  crew  system  design. 

To  understand  the  problem  of  the 
scattered  crew  system  functions,  it  is 
necessary  to  understand  the  role  the 
WBS  has  in  shaping  the  management 
of  the  system  development.  In  prac¬ 
tice,  once  the  WBS  has  been  defined, 
the  management  organizations  of  both 
the  military  and  the  contractor  are 
changed  to  be  consistent  with  the 
WBS.  Since  the  WBS  Level-3  elements 
are  the  major  products  to  be  devel¬ 
oped  and  delivered,  industry  re-orga¬ 
nizes  into  departments  that  correspond 
to  each  of  these  products,  with  a 
separate  department  head  responsible 
to  the  program  manager  for  those 
specific  Level-3  products.  Since  the 
WBS  model  has  no  Level-3  element  for 
crew  system,  industry  has  no  depart¬ 
ment  head  responsible  for  the  crew 
system.  The  task  of  integrating  the 
crew  system  requires  coordination 
among  several  departments  within  the 
company.  This  coordination  is  further 
Continued  on  page  4 


Figure  2.  Ihe  modern  F-22  prototype  cockpit  has  multifunction  displays.  The  push  buttons 
around  these  displays  do  not  have  permanent  labels,  for  their  functions  change  to  match 
the  information  displayed.  7 his  uncluttered  appearance  conceals  a  mind-boggling 
amount  of  information  and  control  options. 
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hindered  because  many  of  the  WBS 
elements  are  subcontracted  to  other 
companies,  with  the  prime  contractor 
serving  as  the  sole  coordinating  agent. 
So,  a  change  made  in  one  department 
may  adversely  affect  the  crew  system 
function  in  another  department  without 
the  other  department’s  being  aware  of  a 
problem  until  it  is  too  late  to  correct. 

So  far,  requests  to  modify  the  WBS 
standard  to  consolidate  and  integrate 
the  crew  system  into  a  single  Level-3 
WBS  element  have  fallen  on  deaf  ears. 
While  most  of  the  engineering  commu¬ 
nity  supports  this  proposal,  it  is  vehe¬ 
mently  opposed  by  the  cost  accoun¬ 
tants  who  promulgate  the  standard, 
because  it  would  min  their  cost  trace- 
ability  and  prediction  models.  This  is  a 
major  change,  for  it  involves  more  than 
adding  a  new  element  called  “Crew 
System”;  it  also  involves  removing  those 
functions  from  the  other  elements. 
Additionally,  this  proposal  would  cause 
a  significant  re-organization  of  industry, 
removing  some  of  the  traditional  re¬ 
sponsibilities  from  these  department 
managers. 

When  the  Air  Force  begins  to  acquire 
a  new  aircraft  or  make  a  major  modifi¬ 
cation  to  an  existing  aircraft,  a  System 
Program  Office  (SPO)  is  established  by 
bringing  members  of  various  disciplines 
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together  as  a  team.  These  SPOs  are 
located  at  Wright-Patterson  AFB  to  be 
near  the  research  and  development 
expertise  centered  in  the  laboratories 
also  located  there.  This  SPO  team 
translates  the  operational  requirements 
into  a  contract  and  later  manages  that 
contract.  Typically,  the  Air  Force  con¬ 
tracts  with  industry  for  aircraft  design 
and  production.  The  official  involve¬ 
ment  of  military  personnel  in  the  pro¬ 
cess  is  monitoring  industry’s  efforts. 

To  implement  the  Integrated  Product 
Team  (IPT)  approach  to  system  devel¬ 
opment,  the  Air  Force’s  ongoing  F-22 
program  has  made  a  radical  departure 
from  the  WBS  aircraft  model  in  MIL- 
STD-881.  Using  its  prerogative  to  “tai¬ 
lor”  the  model  WBS,  the  F-22  SPO 
completely  overhauled  it  into  eight  level- 
3  elements,  one  for  each  of  the  IPTs, 
one  of  which  is  the  Cockpit  System  IPT. 
The  Cockpit  System  Element  is  subdi¬ 
vided  into  five  level-4  elements:  Pilot- 
Vehicle  Interface  (PVI),  Aircrew  Station 
Accommodations,  Escape,  Life  Support, 
and  Canopy.  The  F-22  program  did  not 
make  a  total  break  with  tradition,  how¬ 
ever,  for  part  of  the  crew  system  is  in 
another  level-3  element,  Avionics,  which 
contains  the  avionics  control  and  dis¬ 
play  hardware.  Notwithstanding  this 
one  exception,  the  F-22  program  is  the 
first  military  program  to  attempt  such  a 
high  level  of  integration  of  the  crew 
system  design  activities.  The  results  to 
date  indicate  this  approach  to  be  far 
superior  to  the  traditional  WBS  model, 
providing  high  visibility  to  crew  system 
issues  and  getting  problems  resolved  in 
favor  of  the  pilot.  The  creation  of  a 
unified  crew  system  design  team  to 
address  all  crew  system  issues  marks  an 
advance  in  the  design  process.  The 
F-22  SPO  believes  that  IPTs  are  effec¬ 
tive,  and  their  use  will  likely  continue 
and  spread  to  other  programs. 

Between  1984  and  1992,  the  Paul  M. 
Fitts  Human  Engineering  Division  spon¬ 
sored  seven  research  and  development 
contracts  involving  five  major  aircraft 
companies,  several  avionics  companies, 
and  other  specialists.  Products  of  that 
work  were  a  formal,  integrated  Crew 
System  Design  Process  (CSDP)  with 


activities  and  procedures  to  highlight 
the  crew  system  as  a  distinct  design 
discipline,  and  a  spectrum  of  computer 
tools  to  serve  the  CSDP.  The  Crew- 
Centered  Cockpit  Design  (CCCD)  project 
continues  to  develop  this  technology  to 
support  the  design  of  new  crew  systems 
and  upgrades  for  existing  crew  systems. 

The  CSDP  currently  has  about  120 
activities,  most  supported  by  separate 
software  design  tools.  It  is  beyond  the 
scope  of  this  paper  to  describe  all  of 
them.  These  activities  are  divided  into 
five  categories:  Program  Planning/ 
Scheduling,  Requirements  Analysis  and 
Predesign,  Crew  System  Analysis,  Crew 
System  Design,  and  Crew  System  Evalu¬ 
ation.  The  crew  system  design  category 
accounts  for  the  majority  of  the  activi¬ 
ties.  The  CSDP  tool  set  is  directly  linked 
into  a  generic  crew  system  simulator, 
which  is  reconfigurable  without  sophis¬ 
ticated  programmer  support.  Built  with 
object-oriented  software,  it  allows  a 
journeyman  programmer  to  modify  or 
even  create  a  new  display  for  the  sys¬ 
tem.  The  simulator  is  an  integral  part  of 
the  CSDP  tool  set,  allowing  the  various 
analyses  and  evaluations  to  share  data. 

CSERIAC  has  participated  in  the  CCCD 
project  almost  from  the  beginning,  is 
currently  helping  improve  the  tools  and 
technology,  and  will  make  these  tools 
available  to  industry  when  they  are 
completed.  This  promising  technology 
is  beginning  validation,  with  a  comple¬ 
tion  in  1997.  Two  of  five  validation 
applications  have  been  completed  and 
the  third  is  in  progress.  These  include 
crew  systems  with  different  crew  sizes 
and  operational  missions. 

The  F-22  program’s  use  of  IPTs  forced 
the  creation  of  a  new  aircraft  model  for 
the  WBS.  Its  success  proves  the  efficacy 
of  an  integrated  crew  system  as  a  level- 
3  element.  We  hope  this  momentum 
can  influence  another  revision  of  MIL- 
STD-881  to  include  an  integrated  crew 
system.  • 

Joe  W.  McDaniel,  Ph.D.,  CPE,  is  an 
Industrial  Engineer  with  the  Design 
Technology  Branch,  Fitts  Human  Engi¬ 
neering  Division,  Armstrong  Labora¬ 
tory,  Wright-Patterson  AFB,  OH. 
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he  design  of  aircraft  has 
(  evolved  over  time,  requir¬ 
ing  greater  integration, 
consistency,  and  compat¬ 
ibility  between  the  crew  system  and 
the  capabilities  of  the  operator.  De¬ 
sign  and  program  management  must 
support  this  changing  environment. 

Sometimes  this  has  not  happened, 
especially  when  standards  and  guide¬ 
lines  developed  long  ago  neglect 
changing  environments.  In  the  feature 
article  of  this  issue  of  Gateway ,  Dr.  Joe 
McDaniel  of  the  Armstrong  Laboratory 
discusses  this  problem  with  a  particu¬ 
lar  standard  and  offers  a  potential 
solution,  one  already  being  used  by 
the  F-22  System  Program  Office. 


Also  in  this  issue,  we  present  an¬ 
other  summary  of  a  presentation  from 
the  Armstrong  Laboratory  Human  En¬ 
gineering  Division  Colloquium  Series: 
The  Human-Computer  Interface.  This 
time  our  guest  was  Dr.  Bonnie  John  of 
Camegie-Mellon  University,  who  spoke 
on  “Applying  Psychology  to  the  De¬ 
sign  of  Computer  Systems.”  A  col¬ 
league  here  at  the  Armstrong  Labora¬ 
tory,  Dr.  Mike  Vidulich,  provides  a 
synopsis  of  Dr.  John’s  presentation, 
and  I  follow  that  with  some  excerpts 
from  a  conversation  I  had  with  her. 

Related  to  the  topic  of  integrated 
crew  stations  as  discussed  in  Joe 
McDaniel’s  feature  article,  CSERIAC 
has  been  supporting  the  Crew-Cen¬ 


tered  Cockpit  Display  (CCCD)  pro¬ 
gram  for  several  years.  CSERIAC  Project 
Manager  and  Senior  Design  Engineer 
Mark  Detroit  explains  CCCD’s  goals 
and  achievements  in  this  issue. 

Previously  in  Gateway ,  the  highest 
level  of  CSERIACs  technical  inquiry 
services,  the  Technical  Area  Task  (TAT), 
was  defined,  with  a  promise  to  pro¬ 
vide  some  examples  of  various  TATs 
managed  by  CSERIAC.  Concluding 
this  issue  is  an  article  written  by 
CSERIAC  Project  Manager  and  Human 
Factors  Analyst  Laurie  Quill  in  which 
she  describes  a  TAT  being  conducted 
to  find  ways  of  improving  flightline 
maintenance  procedures.  This  work  is 

Continued  on  page  6 
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being  done  for  the  Human  Resources 
Division,  another  component  of  the 
Armstrong  Laboratory. 

We  are  planning  a  new  series  of 
Gateway  articles,  which  will  feature 
various  ergonomics  research  facilities 
and  programs  throughout  the  world. 
Gateway  now  has  a  circulation  of 


over  10,000  and  is  sent  to  3 6  countries; 
we  think  these  articles  would  make 
interesting  reading  for  this  large 
and  diverse  audience.  If  you  would 
like  us  to  consider  your  organization 
for  inclusion  in  this  series  or  just  want 
more  information,  please  contact  our 
Gateway  Editor,  Jeff  Landis.  He  can 


be  reached  by  any  of  the  methods 
described  on  the  back  cover  of  this 
newsletter.  • 

Reuben  “Lew”  Hann,  Ph.D.,  is  the  Con¬ 
tracting  Officer’s  Technical  Representative 
(COIR)  who  serves  as  the  Government 
Manager  for  the  CSER1AC  Program. 


Calendar 


September  14-16,  1995 
Washington,  DC,  USA 

Work,  Stress,  and  Health  ’95:  Creating 
Healthier  Workplaces.  Contact  Lynn  A. 
Letoumeau,  American  Psychological 
Association,  750  First  St.  NE,  Washington,  DC 
20002-4242;  (202)  336-6124,  fax  (202)  336- 
6117. 


September  18-22,  1995 
Boston,  MA,  USA 

Industrial  Ergonomics:  Human  Factors  in 
Occupational  Health  and  Safety  short  course. 
Contact  Nicole  Costa,  Harvard  School  of 
Public  Health,  Office  of  Continuing 
Education,  677  Huntington  Ave.,  LL-23, 
Boston,  MA  02115-6023;  (617)  432-1171,  fax 
(617)432-1969-  Email: 
contedu@sph.harvard.edu 


October  9-13, 1995 
San  Diego,  CA,  USA 

Human  Factors  and  Ergonomics  Society  39th 
Annual  Meeting,  “Designing  for  the  Global 
Village.”  Hosted  by  the  San  Diego  Chapter. 
Contact  HFES,  PO  Box  1369,  Santa  Monica, 
CA  90406-1369;  (310)  394-2410,  fax  (310) 
394-2410.  Email: 
72133-l474@compuserve.com 


October  16-20,  1995 
Rio  de  Janeiro,  Brazil 

Ergonomics  Design:  Interfaces,  Products, 
Information.  Contact  Stephan  Konz,  Dept,  of 
Industrial  Engineering,  Kansas  State 
University,  Manhattan,  KS  66502;  fax  (913) 
532-7810.  Email:  sh@ksuvm.ksu.edu 


November  14-16,  1995 
Yellow  Springs,  OH,  USA 

A  Short  Course  in  Anthropometry.  This 
course  emphasizes  hands-on  training  in 
anthropometric  measurement  and  provides 
background  lecture  material.  Contact 
Anthropology  Research  Project,  Inc.,  PO  Box 
307,  Yellow  Springs,  OH  45387.  (513)  767- 
7226,  fax  (513)  767-9350. 


February  11-16,  1996 
Fremantle,  Western  Australia 

2nd  International  Conference  on  Fatigue  and 
Transportation:  Education,  Engineering,  and 
Enforcement  Solutions.  Contact  Laurence  R. 
Hartley,  Dept,  of  Psychology,  Murdoch 
University,  Western  Australia  6150.  +61  9 
360  2398,  fax  +61  9  310  9611.  Email: 
Hartley@socs.murdoch.edu.au. 


September  24-28,  1995 
Montreal,  Quebec,  Canada 

2nd  International  Scientific  Conference  on 
Prevention  of  Work-Related  Musculoskeletal 
Disorders,  PREMUS  95-  Organized  by  the 
Institut  de  recherche  en  sante  et  en  securite 
du  travail  du  Quebec  (IRSST)  under  the 
auspices  of  the  Scientific  Committee  on 
Musculoskeletal  Disorders  of  the  International 
Commission  on  Occupational  Health. 

Contact  IRSST,  505,  Boulevarde  de 
Maisonneuve  Ouest,  Montreal,  Quebec, 
Canada,  H3A  3C2;  (514)  288-1551,  fax  (514) 
288-7636. 


October  23-25,  1995 
Quebec  City,  Quebec,  Canada 

27th  Annual  Conference  of  the  Human 
Factors  Association  of  Canada.  Contact  Peter 
Fletcher,  HFAC/ACE,  6519  B  Mississauga  Rd., 
Mississauga,  ON,  Canada  L5N  1A6;  (905) 
567-7193,  fax  (905)  567-7191- 


April  10-12,  1996 
Leicester,  United  Kingdom 

1996  Annual  Conference  of  the  Ergonomics 
Society  to  be  held  at  the  University  of 
Leicester.  Contact  the  Conference  Manager, 
The  Ergonomics  Society,  Devonshire  House, 
Devonshire  Square,  Loughborough, 
Leicestershire  LE11  3DW,  UK.  Telephone 
and  fax  +44  509  234904.  Abstracts  due 
September  22,  1995. 


September  28-29,  1995 
Atlanta,  GA,  USA 

Human-Computer  Interaction  short  course. 
Contact  Dept,  of  Continuing  Education, 
Georgia  Institute  of  Technology,  Atlanta,  GA 
30332-0385;  (404)894-2547.  Email: 
conted@gatech.edu 


October  30-November  2,  1995 
Arlington,  VA,  USA 

Defense  Technical  Information  Center  (DTIC) 
Annua!  Users’  Meeting  and  Training 
Conference.  This  conference  will  address  the 
numerous  types  of  information  available  to 
the  Department  of  Defense  community 
through  the  Internet  as  well  as  from  DTIC 
and  other  government  agencies.  Contact  Julia 
Foscue,  Conference  Coordinator,  Directorate 
of  User  Services,  Special  Programs  Branch, 
Defense  Technical  Information  Center, 
Cameron  Station,  Alexandria,  VA  22304-6145. 
(703)  274-3848. 


Notices  for  the  calendar  should  be  sent  at  least  four  months  in  advance  to: 

CSERIAC  Gateway  Calendar,  AL/CFH/CSERIAC  Bldg  248,  2255  H  Street,  Wright-Patterson  AFB  OH  45433-7022 


April  14-18, 1996 

Vancouver,  British  Columbia,  Canada 

CHI  96.  Conference  on  Human  Factors  in 
Computing  Systems.  Contact  Deborah 
Compere,  CHI  96  Conference  Administrator, 
Conference  and  Logistics  Consultants,  703 
Giddings  Ave.,  Suite  U-3,  Annapolis,  MD 
21401.  (410)  263-5382,  fax  (410)  267-0332. 
Email:  chi96-office@sigchi.acm.org 
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Armstrong  Laboratory  Human  Engineering  Division  Colloquium  Series 

Applying  Psychology  to  the  Design  of  Computer 
Systems 


Bonnie  John 

Synopsis  by  Michael  A.  Vidulich 


Editor’s  note:  Following  is  a  synopsis  of  a 
presentation  by  Dr.  Bonniejohn,  Carnegie - 
Mellon  University,  as  the  fourth  speaker  in 
the  1994  Armstrong  Laboratory  Human 
Engineering  Division  Colloquium  Series: 
The  Human-Computer  Interface.  Th  is  syn  - 
opsis  was  prepared  by  Michael  A.  Vidulich, 
Human  Interface  Technology  Branch,  Fitts 
Human  Engineering  Division,  Armstrong 
Laboratory .  JAL 

Or  John  addressed  the  issue 
of  how  psychology  and 
human  factors  could  con¬ 
tribute  to  the  interface  de¬ 
sign  process.  One  especially  desirable 
way  to  contribute  to  the  design  pro¬ 
cess  would  be  through  the  develop¬ 
ment  of  engineering  models  of  human 
performance.  These  models  would 
ideally  make  quantitative  estimates  of 
the  time  to  learn  tasks,  the  time  to 
perform  tasks,  and  the  number  and 
type  of  errors  that  could  be  expected 
from  a  given  interface  design  in  a 
specified  task  domain.  These  models 
would  then  be  tools  that  could  be  used 
by  system  designers  to  identify  prom¬ 
ising  interface  concepts  at  the  earliest 
possible  stage  of  a  system  design.  If 
such  tools  were  valid,  considerable 
time  and  expense  could  be  saved  in 
the  design  process. 

To  illustrate  the  potential  of  such 
tools,  Dr.  John  reviewed  her  involve¬ 
ment  in  Project  Ernestine,  which  was 
an  evaluation  of  a  proposed  redesign 
of  the  Toll  and  Assistance  Operators 
(TAO)  interface  conducted  by  NYNEX 
(the  parent  company  of  New  England 
Telephone)  (Gray,  John,  &  Atwood, 
1993).  The  TAOs  are  highly  skilled 
workers  who  handle  many  calls  in  the 
course  of  a  work  shift.  Given  the 


number  of  TAOs  employed  by  NYNEX 
and  the  number  of  calls  handled  by 
each,  it  was  estimated  that  saving  just 
1  second  of  the  average  call-handling 
time  would  save  NYNEX  about  $3 
million  a  year.  Not  surprisingly,  NYNEX 
was  considering  buying  a  new  work¬ 
station  and  application  software  to 
speed  up  the  TAO’s  average  call-han¬ 
dling  time.  Based  on  the  expected 
improvement  in  the  time  taken  to 
display  a  complete  screen-full  of 
information  and  the  reduced  number 
of  keystrokes  required  for  most 
calls,  NYNEX  estimated  a  20%  re¬ 
duction  in  the  average  time  per  call. 
NYNEX  decided  to  test  the  proposed 
new  system  in  an  extensive  field 
evaluation. 

Independent  of  the  field  evaluation, 
Dr.  John  and  her  colleagues  conducted 
an  analytical  evaluation  of  the  new 
and  old  interface  designs  using  Card, 
Moran,  and  Newell’s  (1983)  Goals, 


Operators,  Methods,  and  Selection 
(GOMS)  modeling  methodology.  Con¬ 
trary  to  the  intuitions  of  everyone 
involved  in  the  redesign  of  the  TAO 
interface,  the  GOMS  analysis  predicted 
that  performance  with  the  new  inter¬ 
face  would  be  somewhat  worse  than 
with  the  old  interface.  This  was  be¬ 
cause  the  new  interface  benefits  gen¬ 
erally  appeared  in  sub-components  of 
the  TAOs  task  that  were  not  on  the 
critical  time-line.  Meanwhile,  there 
was  a  predicted  slowing  of  some  criti¬ 
cal  time-line  events. 

The  model  predicted  that  overall 
performance  would  be  about  3% 
slower  with  the  new  interface,  and  this 
was  exactly  what  was  found  in  the 
field  evaluation  (see  Fig.  1).  Dr.  John 
contended  that  the  model’s  analysis 
was  essential  for  providing  an  expla¬ 
nation  of  the  counter-intuitive  results 
of  the  field  study.  Taken  together,  the 
Continued  on  page  8 


Figure  1 .  The  GOMS  Model  predicted  that  with  the  new  interface  design,  some  tasks  would 
be  performed  more  slowly  while  others  would  be  performed  more  quickly.  Out  of  15  tasks 
analyzed,  only  two  were  predicted  to  result  in  improved  performance. 
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field  study  results  and  the  GOMS  mod¬ 
eling  analysis  convinced  NYNEX  to 
reject  the  new  interface  design. 

Dr.  John  concluded  that  such  engi¬ 
neering  models  of  human  performance 
are  useful  design  and  evaluation  tools. 
The  proper  use  of  such  tools  should,  in 
the  future,  help  designers  avoid  the 
production  of  sub-optimal  interface 
designs.  # 
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Request  for  Topics 
For 

State-of-the-Art  Reports  (SOARS) 


Scenes  from  the  Armstrong  Laboratory 
Human  Engineering  Division  Colloquium  Series: 


Dr.  Bonnie  John,  Camegie-Mellon  University,  discussing  the  evaluation  of  interface 
designs .  Photo  by  Larry  Burgess,  University  of  Dayton. 


CSERIAC  makes  every  effort  to  be 
sensitive  to  the  needs  of  its  users. 
Therefore,  we  are  asking  you  to  sug¬ 
gest  possible  topics  for  future  SOARS 
that  would  be  of  value  to  the  Human 
Factors/Ergonomics  community.  Pre¬ 
vious  SOARs  have  included  Hypertext: 
Prospects  and  Problems  for  Crew 
System  Design  by  Robert  J.  Glushko, 
and  Three  Dimensional  Displays :  Per¬ 
ception,  Implication,  Applications  by 
Christopher  D.  Wickens,  Steven  Todd, 
&  Karen  Seidler.  Your  input  would  be 
greatly  appreciated. 

Send  your  suggestions  and  other 
replies  to: 

CSERIAC  Program  Office 
AL/CFH/CSERIAC  Bldg  248 
ATTN: Dr.  Ron  Schopper, 

Chief  Scientist 
2255  H  Street 
Wright-Patterson  AFB  OH 
45433-7022 


Dr.  John  ponders  a  question  from  the  audience.  Photo  by  Larry  Burgess,  University 
of  Dayton. 
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Armstrong  Laboratory  Human  Engineering  Division  Colloquium  Series 

A  Conversation  with  Bonnie  John 

Reuben  L.  Hann 


Editor's  note:  Following  is  an  edited 
transcipt  of  a  conversation  with  Dr.  Bonnie 
John ,  Camegie-Mellon  University,  as  the 
fourth  speaker  in  the  1994 Armstrong  labo¬ 
ratory  Human  Engineering  Division 
Colloquium  Series:  Ike  Human-Computer 
Interface.  'Ike  interviewer  was  Dr.  Lew 
Hann,  CSERIAC  COTR.  JAL 

OSERIAC:  I  see  your  edu¬ 
cational  background  is  in 
engineering.  How  did  you 
eventually  end  up  in  “human  factors?” 

Dr.  John:  Actually,  I  got  into  engi¬ 
neering  while  in  high  school.  This  was 
in  New  York,  where  the  schools  were 
“zoned.”  My  zoned  high  school  had  a 
bad  reputation  for  drugs  and  other 
serious  problems.  The  only  way  to 
escape  was  for  me  to  go  into  “pre¬ 
engineering”  or  four  years  of  Latin.  I 
chose  the  former  because  I  thought  it 
would  be  more  fun.  I 
continued  on  through 
graduate  school  with 
a  major  in  Mechanical 
Engineering.  I  worked 
as  a  mechanical  engi¬ 
neer  at  Bell  Labs, 
where  I  designed 
boxes  around  other 
people’s  circuits,  tak¬ 
ing  into  consideration 
such  things  as  heat 
transfer,  power,  and 
so  forth.  Then  I  found 
myself  on  a  commit¬ 
tee  looking  at  the  problem  of  telecon¬ 
ferencing  using  a  single  telephone 
line,  where  you  wanted  to  use  not  only 
voice,  but  also  FAX,  electronic  black¬ 
board,  and  slow-scan  TV.  The  ques¬ 
tion  was  how  to  decide  which  of  these 
had  control  of  that  single  line  at  any 
moment.  The  committee  was  strug¬ 


gling  with  determining  the  communi¬ 
cation  protocols  suitable  for  this  situa¬ 
tion.  It  was  really  an  enjoyable  project. 
Then  my  department  head  yanked  me 
off  the  committee,  telling  me  that  me¬ 
chanical  engineers  don’t  do  that  sort  of 
work.  I  asked  him  who  does.  He  told 
me  that  was  the  work  of  system  engi¬ 
neers.  So  I  decided  that’s  what  I 
wanted  to  do.  I  transferred  into  a 
Systems  Engineering  department, 
where  I  wrote  specifications  for  the 
Merlin  telephone  system.  I  am  very 
proud  of  that  work;  it  was  the  only 
thing  which  sold  well  for  a  while  after 
the  breakup  [of  the  Bell  Telephone 
Company].  It  is  still  popular  after  ten 
years. 

CSERIAC:  This  was  a  Systems  Engi¬ 
neering  department;  how  did  you  get 
involved  with  more  traditional  human 
factors  issues? 


in  understanding  whether  what  I  was 
designing  was  going  to  be  easy  for 
people  to  use.  I  took  a  lot  of  courses 
in  Human  Factors  at  Stevens  Tech — 
almost  enough  for  a  Master’s  degree, 
in  fact.  In  my  work,  it  seemed  that  at 
every  turn,  the  most  important  ques¬ 
tion  was  always  whether  the  system 


was  going  to  be  usable  by  people.  My 
mechanical  engineering  training  was 
not  helping  me  to  answer  that  ques¬ 
tion.  But  I  also  found  that  the  tradi¬ 
tional  experimental  psychology  and 
human  factors  training  was  not  help¬ 
ing  as  much  as  I  wanted  it  to  when 
dealing  with  the  cognitive  aspects  of 
the  problem.  So  I  went  to  get  a  degree 
in  Cognitive  Psychology  at  Carnegie- 
Mellon  University  (CMU). 

I  was  tiying  to  determine  what  cre¬ 
dentials  I  would  need  as  a  professional 
to  try  to  solve  these  kinds  of  problems. 
It  looked  to  me  at  the  time  that  nobody 
listened  to  you  unless  you  had  a  Ph.  D. 
in  psychology,  so  I  chose  that  as  the 
credential  to  get.  I  chose  CMU  be¬ 
cause  it  was  immediately  apparent  that 
there  was  excellent  communication 
and  cooperation  between  the  various 
departments — Psychology  and  Com¬ 
puter  Science,  for  instance.  And,  the 
facilities  were  impres¬ 
sive;  at  the  time — this 
was  1982 — the  Psy¬ 
chology  department 
had  six  VAX  comput¬ 
ers — more  than  most 
university  computer 
science  departments. 
I  decided  this  was  the 
place  to  pursue  a  de¬ 
gree  in  Cognitive  Psy¬ 
chology,  where  my 
goal  was  the  study  of 
human-computer  in¬ 
teraction. 

CSERIAC:  I  see  that  CMU  has  estab¬ 
lished  a  new  institute  in  the  area  of 
Human-Computer  Interaction. 

Dr.  John:  Yes,  there  are  faculty 
from  8  or  10  different  organizations 
around  campus  who  all  work  on  differ¬ 
ent  aspects  of  human-computer  interac- 
Continuecl  on  page  10 


Dr.  John:  Well,  I  became  interested 

“I  was  trying  to  determine  what  credentials 
I  would  need  as  a  professional  to  try  to  solve 
these  kinds  of  problems.  It  looked  to  me  at 
the  time  that  nobody  listened  to  you  unless 
you  had  a  Ph.D.  in  psychology,  so  I  chose 
that  as  the  credential  to  get.” 
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tion.  So  there  are  computer  scientists, 
psychologists,  people  from  social  and 
decision  sciences,  from  the  Graduate 
School  of  Industrial  Administration,  from 
the  Industrial  Design  Department, 
people  from  the  Software  Engineering 
Institute,  from  robotics,  and  others  I 
may  have  forgotten.  There  are  about  30 
or  so  faculty  who  have  been  meeting 
periodically  for  the  past  year  to  bring 
together  a  research  institute  that  will 
facilitate  our  working  together  and  hav¬ 
ing  joint  projects. 

CSERIAC:  How  do  these  projects 
come  about?  Do  they  come  in  from  the 
outside? 

Dr.  John:  Yes,  right  now  the  Institute 
is  a  loose  amalgamation  of  independent 
researchers  who  cooperate  when  the 
science  is  “right.”  We  don’t  force  any¬ 
thing.  As  they  get  to  know  each  other 


better,  they  will  know  when  they  should 
be  making  a  joint  proposal  to  outside 
agencies,  such  as  government  or  indus¬ 
tries  involved  in  areas  like  software  and 
telecommunications. 

CSERIAC:  I  have  asked  many  of  my 
guests  what  kind  of  research  they  would 
undertake  if  the  problem  of  financial 
resources  were  removed.  What  kind  of 
problems  would  you  tackle  if  you  had 
no  restriction? 

Dr.  John:  I  would  study  the  issue  of 
— how  do  people  learn  something  new? 
How  do  they  use  what  they  already 
know  to  go  into  a  new  computer  system 
and  learn  it?  By  “learn  it”  I  mean — how 
do  they  use  prior  knowledge  to  guide 
their  problem  solving?  How  do  they 
formulate  problems  in  the  first  place? 
How  do  they  explore  the  problem  to 
find  out  how  this  knowledge  can  help 


to  reach  the  goal?  How  do  they  retain 
that  information?  How  do  they  react 
with  it  if  there  is  time  pressure?  So  I 
am  interested  in  what  happens  with  a 
new  system.  How  do  you  explore  it, 
problem-solve  with  it?  How  does  it 
help  guide  you,  and  how  do  you  learn 
the  information  needed  to  perform 
satisfactorily? 

There  is  a  rule-of-thumb  in  the  indus¬ 
trial  software  business  that,  in  any  com¬ 
plex  system,  any  one  person  only  knows 
about  10%  of  the  functionality  of  the 
system.  And  it’s  a  different  10%  for  each 
person.  So  how  does  that  happen? 
How  do  people  get  into  a  particular 
“comer”  regarding  what  they  know, 
and  how  can  you  help  break  them  out, 
so  they  can  learn  more.  All  these  issues 
of  learning  and  problem-solving — that’s 
where  I  would  put  my  effort.  • 


Behind  Human  Error 

Cognitive  Systems,  Computers,  and  Hindsight 

David  D.  Woods,  Leila  J.  Johannesen,  Richard  I.  Cook,  &  Nadine  B.  Sailer 

The  Ohio  State  University 


W  SYSTEM  ERGONOMICS  INFORMATION  ANALYSIS  CENTER 


SOAR 


Stale-of-lhe  Art  Report 

Behind  Human  Error: 
Cognitive  Systems, 
Computers,  and  Hindsight 


Behind  Human  Error:  Cognitive  Systems, 
Computers,  and  Hindsight  (Woods, 
Johannesen,  Cook,  and  Sarter,  1994). 


ccident  investigations  have  often  found  operators  of  complex 
systems  to  be  points  of  failure,  and  hence  the  perception  exists 
that  there  is  a  human  error  problem.  This  view  turns  out  to  be 
too  simplified  to  allow  us  to  learn  from  incidents  and  failures.  To  learn 
about  the  nature  of  system  failure,  one  must  go  behind  human  error  by 
seeing  error  not  as  an  end  point,  but  as  the  starting  point  for  investigation. 
A  new  state-of-the-art  report  (SOAR)  from  CSERIAC  investigates  what 
lies  behind  human  error.  It  explains  how  outcome  knowledge  biases  our 
attribution  of  error.  It  shows  how  cognitive  system  factors  play  a  role  in 
accidents  and  illustrates  the  importance  of  strategic  tradeoffs  and 
conflicting  goals  faced  by  system  operators.  It  focuses  especially  on  how 
the  design  of  computers,  automation,  and  other  new  technology  affects 
the  potential  for  system  failure. 

Price:  $39  plus  shipping .  To  order ,  contact  the  CSERIAC  Program 
Office  at  (513)  255-4842  or  DSN  785-4842. 
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Crew-Centered  Cockpit  Design  Program 

Mark  Detroit 
Cindy  Martin 
Capt  Steve  Beyer 


ith  current  acquisition  re¬ 
form  initiatives  and  de¬ 
creasing  military  budgets, 
it  has  become  increasingly 
important  to  ensure  cost-effective  de¬ 
velopment  of  weapon  systems.  In 
response  to  this  need,  the  Crew-Cen¬ 
tered  Cockpit  Design  (CCCD)  Project 
is  developing  a  structured  process  and 
tool  set  to  improve  the  design,  analy¬ 
sis,  and  testing  of  cockpits  (see  Fig.  1). 
The  CCCD  Project  is  an  advanced 
technology  development  project  at 
the  Armstrong  Laboratory  Fitts 
Human  Engineering  Division.  Under 
the  CCCD  Field  Demonstration  Con¬ 
tract,  Veda  Incorporated  is  working 
with  the  CCCD  Project  Office  to  en¬ 
hance  and  validate  a  new  Crew-Cen¬ 
tered  System  Design  Process  (CSDP) 
and  a  Cockpit  Design  System  (CDS) 
tool  set  to  meet  this  need.  In  parallel 
with  developing  the  process  and  tools, 
CSERIAC  has  been  working  with  the 
CCCD  Project  Office  to  evaluate  the 
tools  and  to  provide  ancillary  support. 

The  Crew-Centered  System  Design 
Process  (CSDP)  is  a  detailed  descrip¬ 
tion  of  the  activities  that  are  necessary 
for  cockpit  design.  It  is  patterned  after 
time-honored  practices  within  the 
aircraft  industry,  but  adds  computer 
support  and  a  stronger  focus  on  hu¬ 
man-centered  design.  These  CSDP 
activities  are  electronically  accessible 
through  a  software  interface  tool 
known  as  the  Design  Traceability 
Manager  (DTM).  The  DTM  software 
represents  a  technology  advance, 
because  previous  design  practices 
were  neither  implemented  through 
software  nor  did  they  have  a  means  to 
capture  the  progression  of  cockpit 
design  decisions  (useful  for  managing 
crew  system  change).  For  each  of  the 


classical  weapon  system  acquisition 
phases  (such  as  Concept  Exploration, 
Demonstration  &  Validation,  and 
Engineering  &  Manufacturing  Devel¬ 
opment)  the  DTM  user  can  access  and 
invoke  CSDP  activities  which  are 
organized  into  four  major  design  cat¬ 
egories: 

■  Program  Planning 

■  Requirements  Analysis  and 
Predesign 

■  Design 

■  Evaluation 

Version  4  of  the  CSDP,  scheduled  for 
completion  in  January  1996,  is  being 
developed  as  part  of  the  Field  Demon¬ 
stration  contract.  This  version  will  (1) 
reflect  current  acquisition  reform  ini¬ 
tiatives,  (2)  incorporate  government 
standards  established  in  MIL-STD-1776, 
including  the  Crew  System  SEMS  (Sys¬ 
tem  Engineering  Master  Schedule),  and 
(3)  consolidate  work  done  under  pre¬ 
decessors  to  the  current  contract. 

The  CDS  toolset  comprises  an  as¬ 
semblage  of  customized  software,  com¬ 
mercial  off-the-shelf  software,  software 
products  created  by  other  Air  Force 
projects,  and  a  real-time  engineering 
cockpit  simulator.  Customized  soft¬ 
ware  includes  the  Design  Traceability 
Manager  and  the  Timeline  Manage¬ 
ment  Tool.  Both  tools  are  being  re¬ 
engineered  to  run  on  an  IBM-compat¬ 
ible  PC  to  enhance  their  accessibility  to 
the  users.  Air  Force  tools  include  the 
Tool  for  Automated  Knowledge  Engi¬ 
neering  (TAKE)  and  the  CSERIAC-de- 
veloped  Requirements  Translator  Tool 
(RTT). 

An  adjunct  to  the  CDS  is  a  new  tool 
that  supports  crew  system  Test  &  Evalu¬ 
ation  functions.  The  Test  Planning, 
Analysis,  and  Evaluation  System  (Test 


PAES)  is  an  interactive  tool  that  helps 
to  plan  and  perform  cockpit  evalua¬ 
tion  in  a  flight  test.  Test  PAES  includes 
a  structured  test  and  evaluation 
process  (analogous  to  the  Crew-Cen¬ 
tered  System  Design  Process),  struc¬ 
tured  test  procedures,  a  visualization 
system  to  play  back  time-synchro¬ 
nized  multi-media  data  (collected  in  a 
flight  test)  for  analysis  and  debriefing, 
and  an  array  of  other  software  tools 
which  have  proved  useful  to  the  test 
community. 

Applications 

The  Crew-Centered  Cockpit  Design 
Project  is  validating  its  computer 
tools  and  design  process  by  applying 
them  to  typical  Air  Force  cockpit 
projects.  Two  of  five  planned  applica¬ 
tions  have  been  completed.  The  first 
application  examined  the  effects  of 
redesigning  the  single-place  F-l6  cock¬ 
pit  for  a  new  operational  mission, 
tactical  reconnaissance.  The  second 
application  examined  an  upgrade  of 
the  Fire  Control  Operator’s  crew  sta¬ 
tion  in  the  multi-place  AC-130H  Gun- 
ship.  Both  applications  successfully 
showed  that  use  of  the  process  and 
tool  set  can  lead  to  measurably  im¬ 
proved  crew  station  designs  (i.e.,  im¬ 
proved  crew  performance,  mission 
performance,  and  reduced  workload 
in  critical  mission  segments).  Better 
performance  was  first  predicted  ana¬ 
lytically  and  then  verified  through  pi¬ 
loted  simulation. 

In  contrast,  Test  PAES  has  com¬ 
pleted  developmental  testing  and  been 
furnished  to  flight  test  users  for  evalu¬ 
ation  in  their  own  operational  envi¬ 
ronments.  Beta  test  sites  have  been 

Continued  on  page  12 
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established  at  the  Combined  Test 
Forces  (for  F-15,  F-l6,  F-22,  F-117,  B- 
1,  B-2,  C-17,  AC-130,  and  U-2  aircraft, 
among  others),  at  various  Air  Force, 
Navy,  Army,  and  other  test  agencies. 

Facilities/Resources 

The  equipment  and  software  that 
comprise  the  CDS  tool  set  reside  in  a 
research  laboratory  within  the  Human 
Engineering  Division.  The  CDS  is 
located  in  Building  248.  It  consists  of 
86  hardware  components  and  137 
computer  software  programs,  all  linked 
via  computer  network.  The  comput¬ 
ing  environment  includes  Silicon 
Graphics  workstations  and  IBM-com¬ 
patible  personal  computers.  The  Test- 
PAES  resides  on  a  single  personal 
computer.  The  CCCD  lab  also  in¬ 
cludes  a  real-time  cockpit  simulator 
that  can  be  reconfigured  both  in  hard¬ 
ware  and  software,  for  testing  the 
effects  of  the  cockpit  design  changes 
relative  to  the  baseline  cockpit.  In 
this  manner,  the  performance  predic¬ 
tions  from  the  CCCD  analysis  tools  can 


be  confirmed  by  measured  perfor¬ 
mance  from  real-time,  manned  simu¬ 
lator  testing. 

Both  the  CDS  and  Test-PAES  will  be 
extended  in  follow-on  work  for  crew- 
centered  applications  to  exploit  the 
emerging  capabilities  offered  by  Ad¬ 
vanced  Distributed  Simulation  and 
Battle  Management  Command  and 
Control,  building  on  the  proven  suc¬ 
cesses  of  this  CCCD  technology,  al¬ 
ready  demonstrated  for  aircraft  cock¬ 
pit  design. 

Products 

The  CCCD  Project,  its  development 
progress,  and  products  have  been 
described  in  numerous  technical  re¬ 
ports,  journal  articles,  conference  pro¬ 
ceedings,  and  informational  briefings, 
both  nationally  and  internationally. 
The  Project  has  been  performed 
through  more  than  ten  research  and 
development  contracts,  involving 
over  20  companies,  including  the 
direct  involvement  of  five  aircraft  manu¬ 
facturers.  • 


Mark  Detroit  is  a  Senior  Design  Engineer 
with  CSERIAC,  Cindy  Martin  is  the  Senior 
Human  Factors  Engineer  for  CCCD  with 
Veda,  and  Air  Force  Capt  Steve  Beyer 
is  the  Assistant  Project  Manager  for  the 
CCCD  Project,  Design  Technology  Branch, 
Fitts  Human  Engineering  Division, 
Armstrong  Laboratory . 


Human  Factors  & 
Ergonomics  Society 
39th  Annual  Meeting 

Designing  for  the 
Global  Village 

The  Annual  Meeting  will 
take  place  October  9-13, 
1995,  at  the  Sheraton  Harbor 
Island  Resort,  San  Diego,  CA. 

Throughout  the  week  the 
meeting  will  feature  hands- 
on  workshops  geared  toward 
professionals  at  all  levels  as 
well  as  more  than  100  tech¬ 
nical  sessions  on  a  broad 
range  of  ergonomics  related 
topics. 

Attendees  will  also  have 
the  chance  to  browse  book, 
service,  and  product  exhib¬ 
its;  tour  San  Diego-area 
technical  and  research 
facilities;  and  attend  special 
events.  The  HFES  Placement 
Service  will  be  available  to 
help  match  job  seekers  and 
employers. 


Figure  1.  A  conceptual  representation  of  the  CCCD  process 
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CSERIAC  Technical  Area  Tasks 

Integrating  Aircraft  Maintenance  Systems:  A  Case  Study 

David  Kancler 
Laurie  Quill 
Joy  Fasnacht 


On  earlier  issue  of  Gate¬ 
way  (Vol.  VI,  No.  1)  pro¬ 
vided  an  overview  of  the 
CSERIAC  Technical  Area 
Task  (TAT)  as  a  vehicle  which  gives 
customers  the  ability  to  tailor 
CSERIAC’s  services  to  meet  their 
unique  human  factors  needs.  This 
article  will  describe  a  TAT  currently 
under  contract  with  the  Logistics 
Research  Division  of  Armstrong 
Laboratory  (AL/HRG).  For  the  past 
five  years,  CSERIAC  has  supported 
the  Division  with  a  series  of  TATs 
dealing  primarily  with  human- 
computer  inter¬ 
face  issues.  For 
example,  several 
flightline  field 
tests  have  been 
conducted,  in¬ 
cluding  the  Inte¬ 
grated  Mainte¬ 
nance  Informa¬ 
tion  System  field 
test  at  Luke  Air 
Force  Base  in 
the  summer  of 
1994. 

Since  1982,  AL/ 

HRG  efforts  have 
focused  on  the 
transfer  of 
flightline  mainte¬ 
nance  procedures 
from  paper-based 
media  to  inte¬ 
grated,  computer¬ 
ized  systems.  Tra¬ 
ditionally,  paper- 
based  procedure 
systems  have  re¬ 
quired  the  use  of 
a  large  quantity  of 


manuals,  which  maintenance  per¬ 
sonnel  must  sift  through  to  com¬ 
plete  a  particular  job.  The  goal  of 
integrated,  computer-based  systems 
has  been  to  present  the  same 
maintenance  information  on  a  less 
cumbersome,  portable  device. 

Among  the  portable  devices 
tested,  the  laboratory  has  investi¬ 
gated  the  use  of  monocular,  occlud¬ 
ing  eye-pieces  to  enhance  the  mo¬ 
bility  of  maintenance  flightline  per¬ 
sonnel  as  depicted  in  Figure  1.  As 
a  follow-on  to  eye-piece  research 
conducted  at  AL/HRG,  voice 


recognition  was  identified  for 
study  in  conjunction  with  eye-piece 
technology.  Maintenance  environ¬ 
ments  often  require  the  use  of 
both  hands;  therefore,  the  labora¬ 
tory  hypothesized  that  the  combina¬ 
tion  of  voice-recognition  and  eye¬ 
piece  technologies  would  allow 
the  user  to  operate  the  computer 
system  while  performing  aircraft 
maintenance  actions. 

To  accomplish  the  proposed  study, 
a  cooperative  effort  was  required 
among  disciplines  and  organizations. 
Disciplines  required  for  the  study 


Figure  1.  A  monocular  occluding  eye-piece  with  voice  recognition  as  tested  on  an  F-16C  airplane  at  the  Air 
National  Guard  Base ,  Springfield ,  Ohio. 
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Table  1. 

CSERIAC  Tasks  on  the  AL/HRG  TAT 

■  Information  Gathering  and  Analysis 

■  Cognition  Theory  Analysis  and  Application 

■  Test  and  Evaluation  Planning 

■  Guidance  in  Software  Design 

■  Graphical  User  Interface  Design 


included  human  factors  specialists, 
software  engineers,  hardware 
engineers,  and  maintenance  person¬ 
nel.  The  participating  organizations 
included  AL/HRG;  Air  Force  Institute 
of  Technology  (AFIT);  CSERIAC;  RJO 
Enterprises,  Inc.;  Computer  Sciences 
Corporation;  and  RCF  Information 
Systems,  Inc.  With  such  diverse  dis¬ 
ciplines  required  for  this  project, 
collaboration  was  necessary  among 
various  government  and  numerous 
|  contracting  organizations. 

In  the  voice  recognition  study, 
CSERIAC  provided  literature  searches, 
analyses  of  cognitively  based 
theories,  and  recommendations  for 
methods  of  testing  the  integrated 
system.  CSERIAC  also  designed  and 
developed  the  graphical  user  inter¬ 
face  (GUI)  software  module  required 
for  the  presentation  application.  The 
key,  however,  to  CSERIAC’s  support 
in  this  project  was  the  capability 
to  use  and  customize  existing 
resources.  For  example,  reports 
created  by  CSERIAC  for  AL/HRG 
were  used  to  support  literature  re¬ 
views  and  provide  theoretical  bases 
for  the  study.  In  addition,  existing 
software  applications  were  custom¬ 
ized  to  fit  the  requirements  of 
the  voice  recognition  study. 
CSERIAC  also  provided  guidance 
in  experimental  design  for  the  study 
of  voice  recognition  as  a  potential 
means  of  input  in  the  computerized 
flightline  maintenance  environment. 
CSERIAC’s  human  factors  expertise, 
plus  familiarity  with  existing  resources 
(inherent  in  TAT  contracts),  provided  the 
unique  experience  required  to  support 
this  project.  Table  1  summarizes  CSERIAC 
tasks  on  the  AL/HRG  TAT. 

Essential  contributions  to  the 
project  were  provided  by  other 
contractors  and  government  person¬ 
nel  as  well.  RJO  Enterprises,  Inc. 
developed  and  integrated  the 
additional  modules  required  for 
the  presentation  application.  One 
of  these  modules  was  an  expert 
system  for  aiding  in  the  decision¬ 
making  processes  (i.e.,  selection  of 
appropriate  troubleshooting  proce¬ 


dures)  required  on  the  flightline.  RJO 
also  integrated  the  Computer-Off- 
The-Shelf  (COTS)  voice  recognition 
software  with  the  presentation  soft¬ 
ware  application.  Hardware  develop¬ 
ment  was  provided  by  Computer 
Sciences  Corporation  and  RCF  Infor¬ 
mation  Systems,  Inc.  AL/HRG  enlisted 
the  use  of  AFIT  graduate  students  to 
conduct  the  study  in  completion  of 
their  masters’  theses. 

Data  collection  was  conducted 
in  the  spring  of  1995  at  the  Air 
National  Guard  Base  in  Springfield, 
Ohio.  Data  analysis  is  underway.  The 
key  to  the  success  of  this  research 
program  has  been  the  collaboration  of 
personnel  within  the  various  disci¬ 
plines  and  organizations  represented 
in  this  effort. 

Within  the  scope  of  a  long-term 
project,  CSERIAC  TATs  provide  the 
flexibility  to  address  specific  customer 
needs  while  adapting  to  technological 
advances.  In  addition,  the  TAT  struc¬ 
ture  allowed  CSERIAC  personnel  to 
provide  human  factors  expertise  not 
only  to  the  customer,  but  also  to  other 
support  contractors,  including  soft¬ 
ware  and  hardware  developers.  Con¬ 
sequently,  human  factors  expertise 
was  available  to  the  entire  develop¬ 
ment  team,  thereby  improving  the 
quality  of  the  final  product.  • 

David  Kancler  and  Laurie  Quill  are  Hu¬ 
man  Factors  Analysts  with  CSERIAC.  Joy 
Fasnacht  is  a  Programmer/ Analyst  with 
RJO  Enterprises,  Inc.,  Dayton,  OH . 


Correction  & 

New  Product 

In  the  last  issue  of  Gateway ;  the 
feattire  article  on  the  50th  Anniversary 
of  the  Paul  M.  Fitts  Human  Engineering 
Division  mentioned  the  availability  of 
50  Years  of  Human  Engineering:  His¬ 
tory  and  Cumulative  Bibliography  of 
the  Fitts  Human  Engineeering 
Division,  1945-1995 Unfortunately,  the 
wrong  telephone  number  was 
provided  for  those  interested  in 
obtaining  a  copy.  We  apologize  for  this 
mistake  and  ask  interested  readers  to 
contact  CSERIAC  directly  at  (513)  255- 
4842  or  DSN  785-4842  to  obtain  a  copy. 

Meanwhile,  we  are  pleased  to  an¬ 
nounce  that  this  same  work  is  available 
on  a  CD-ROM,  as  well.  Again,  please 
make  your  inquiries  directly  to 
CSERIAC  at  one  of  the  aforementioned 
telelphone  numbers. 


50  Years  of  Human  Engineering: 
History  and  Cumulative  Bibliography 
of  the  Fitts  Human  Engineeering 
Division,  1945-1 995. 
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ICASHE  PUS  lets  you  interactively  simulate  and  experience  these  perception 
and  performance  phenomena: 

Display  Vibration 
Visual  Optics 
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Speech  Intelligibility  in  Noise 
Manual  Control 


Perceived  Motion 
Auditory  Sensitivity 
Sound  Localization 
Visual  Acuity 

Visual  Search  and  Target  Acquisition 

Flicker  Sensitivity 

Visual  and  Auditory  Warning  and  Alerts 

CASHEfVS  uses  hypertext  to  access  the  text,  data,  and  graphics  in  the 
Engineering  Data  Compendium  anti  MIL-STD-1472D  Human  Engineering 
Design  Criteria  for  Military  Systems,  Equipment  and  Facilities. 


QERtAC 

CSERIAC  Program  Office 
AL/CFH/CSERIAC  Bldg  248 
2255  H  Street 
Wrigbt-Patterson  AFB  OH  45433-7022 
(513)255-4042 
EAX  (513)  255-4823 


A  valuable  reference  and  instructional  tool,  CASHEWS 
is  available  for  $400.00  from  CSERIAC.* 


Sponsored  by: 

Armstrong  Laboratory;  Air  Force  Office  of  Scientific  Research;  Army  Research  Laboratory;  Naval 
Command,  Control  and  Ocean  Surveillance  Center;  Defense  Technical  Information  Center;  Federal 
Aviation  Administration;  and  NATO/AGARD 

*Site  license  available  with  five  or  more  copies. 
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CSERIAC 
PRODUCTS  AND 
SERVICES 

CSERIAC's  objective  is  to  acquire, 
analyze,  and  disseminate  timely  infor¬ 
mation  on  crew  system  ergonomics 
(CSE).  The  domain  of  CSE  includes 
scientific  and  technical  knowledge  and 
data  concerning  human  characteris¬ 
tics,  abilities,  limitations,  physiological 
needs,  performance,  body  dimensions, 
biomechanical  dynamics,  strength,  and 
tolerances.  It  also  encompasses  engi¬ 
neering  and  design  data  concerning 
equipment  intended  to  be  used,  oper¬ 
ated,  or  controlled  by  crew  members. 

CSERIAC's  principal  products  and 
services  include: 

■  technical  advice  and  assistance; 


■  customized  responses  to  biblio¬ 
graphic  inquiries; 

■  written  reviews  and  analyses  in 
the  form  of  state-of-the-art  reports  and 
technology  assessments; 

■  reference  resources  such  as  hand¬ 
books  and  data  books. 

Within  its  established  scope,  CSE¬ 
RIAC  also: 

■  organizes  and  conducts  work¬ 
shops,  conferences,  symposia,  and 
short  courses; 

■  manages  the  transfer  of  techno¬ 
logical  products  between  developers 
and  users; 

■  performs  special  studies  or  tasks. 

Services  are  provided  on  a  cost- 
recovery  basis.  An  initial  inquiry  to 
determine  available  data  can  be  ac¬ 
commodated  at  no  charge.  Special 
tasks  require  approval  by  the  Govern¬ 
ment  Technical  Manager. 


To  obtain  farther  information  or  re¬ 
quest  services,  contact: 

CSERIAC  Program  Office 
AI/CFH/ CSERIAC  Bldg  248 
2255  H  Street 

Wright-Patterson  AFB  OH  45433-7022 

Telephone . (513)  255-4842 

DSN . 785-4842 

Facsimile . (513)  255-4823 

Government 

Technical  Manager . (513)  255-8821 

Director:  Mr.  Don  A.  Dreesbach; 
Government  Technical  Manager:  Dr. 
Reuben  L.  Hann;  Associate  Govern¬ 
ment  Technical  Manager:  Ms.  Tanya 
Ellifritt,-  Government  Technical  Direc¬ 
tor:  Dr.  Kenneth  R.  Boff. 
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